Some Observations on the Use of the Extended Kalman Filter as a Recurrent Network Learning Algorithm

The extended Kalman lter (EKF) can be used as an on-line algorithm to determine the weights in a recurrent network given target outputs as it runs. This involves forming an augmented network state vector consisting of all unit activities and weights. This report notes some relationships between the EKF as applied to recurrent net learning and some simpler techniques that are more widely used. In particular, it is shown that making certain simpliications to the EKF gives rise to an algorithm essentially identical to the real-time recurrent learning (RTRL) algorithm. That is, the resulting algorithm both maintains the RTRL data structure and prescribes identical weight changes. In addition, because the EKF also involves adjusting unit activity in the network, it provides a principled generalization of the useful \teacher forcing" technique. Very preliminary experiments on simple nite-state Boolean tasks indicate that the EKF works well for these, generally giving substantial speed-up in number of time steps required for convergence to a solution when compared with the behavior of simpler on-line gradient algorithms like RTRL or truncated backpropagation through time (BPTT). The computational requirements of the EKF for this application are also compared with those of RTRL and BPTT. For a network having n units its storage requirements scale as O(n 4), which exceeds the O(n 3) storage required by RTRL. Interestingly, however, the computation can be organized so that the number of arithmetic operations per time step scales as O(n 4), which is no worse (except by a constant factor) than RTRL. Finally, some speculation is ooered that the EKF and related algorithms may help provide a basis for both gaining a deeper understanding of existing recurrent network learning techniques and creating more computationally attractive algorithms that nevertheless retain the advantages of the EKF, most notably its incrementality and superior convergence speed.